{smcl}
{* 10Aug2007}{...}
{hline}
{hi:help ranktest}{right:(SJ7-4: st0030_3)}
{hline}

{title:Title}

{p2colset 5 17 19 2}{...}
{p2col:{hi:ranktest} {hline 2}}Test the rank of a matrix using the Kleibergen-Paap rk statistic{p_end}
{p2colreset}{...}


{title:Syntax}

{pstd}Full syntax

{p 8 14 2}{cmd:ranktest}
{cmd:(}{it:varlist1}{cmd:)}
{cmd:(}{it:varlist2}{cmd:)}
{ifin}
{weight}
{bind:[{cmd:,} {cmd:partial(}{it:varlist3}{cmd:)}}
{cmd:wald}
{cmdab:all:rank}
{cmdab:full:rank}
{cmdab:null:rank}
{cmdab:rob:ust}
{cmd:cluster}{cmd:(}{it:varname}{cmd:)}
{cmd:bw(}{it:#}{cmd:)}
{cmd:kernel(}{it:string}{cmd:)}
{bind:{cmdab:noc:onstant}]}


{pstd}Version syntax

{p 8 14 2}{cmd:ranktest}{cmd:,} {cmd:version}


{pstd}{cmd:ranktest} may be used with time-series or panel data. 
You must {cmd:tsset} the data before using {cmd:ranktest}; see
{helpb tsset:[TS] tsset}.

{pstd}All {it:varlists} may contain time-series operators; see
{it:{help varlist}}.

{pstd}If {cmd:(}{it:varlist1}{cmd:)} or {cmd:(}{it:varlist1}{cmd:)} contain 
one variable, the parentheses {cmd:()} may be omitted.

{pstd}{cmd:aweight}s, {cmd:fweight}s, {cmd:iweight}s, and {cmd:pweight}s
are allowed; see {help weight}.

{pstd}{cmd:ranktest} is an r-class program.


{title:Contents}

{pstd}{help ranktest##s_description:Description}{p_end}
{pstd}{help ranktest##s_examples:Options}{p_end}
{pstd}{help ranktest##s_macros:Saved results}{p_end}
{pstd}{help ranktest##s_examples:Examples}{p_end}
{pstd}{help ranktest##s_refs:References}{p_end}
{pstd}{help ranktest##s_acknow:Acknowledgments}{p_end}
{pstd}{help ranktest##s_citation:Authors}{p_end}
{pstd}{help ivreg3##s_citation:Citation}{p_end}


{marker s_description}{title:Description}

{pstd}{cmd:ranktest} implements the Kleibergen-Paap (2006) rk test for the rank
of a matrix.  Tests of the rank of a matrix have many practical applications.
For example, in econometrics the requirement for identification is the rank
condition, which states that a particular matrix must be of full-column rank.
Another example from econometrics concerns cointegration in vector
autoregressive (VAR) models; the Johansen trace test is a test of a rank of a
particular matrix.  The traditional test of the rank of a matrix for the
standard (stationary) case is the Anderson (1951) canonical correlations test.
If we denote one list of variables as Y and a second as Z and we calculate the
squared canonical correlations between Y and Z, the LM form of the Anderson
test, where the null hypothesis is that the matrix of correlations or
regression parameters B between Y and Z has rank(B)=r, is N times the sum of
the r+1 largest squared canonical correlations.  A large test statistic and
rejection of the null indicates that the matrix has a rank of at least r+1.  The
Cragg-Donald (1993) statistic is a closely related Wald test for the rank of a
matrix.  Both the Anderson and Cragg-Donald tests require the assumption that
the covariance matrix has a Kronecker form; when this is not so, e.g., when
disturbances are heteroskedastic or autocorrelated, the test statistics are no
longer valid.  

{pstd}The Kleibergen-Paap (2006) rk statistic is a generalization of the
Anderson canonical correlation rank test to the case of a non-Kronecker
covariance matrix.  The implementation in {cmd:ranktest} will calculate rk
statistics that are robust to various forms of heteroskedasticity,
autocorrelation, and clustering.  For a full discussion of the test statistic
and its relationship other test statistics for the rank of a matrix, see
Kleibergen-Paap (2006).

{pstd}The text is applied to Y and Z, where Y={it:varlist1} and
Z={it:varlist2}.  Optionally, a third set of variables X={it:varlist3} can be
partialled-out of Y and Z with the {cmd:partial()} option.  A constant is
automatically partialled-out, unless the user specifies the {cmd:noconstant}
option.  To test if a matrix is rank r+1, the null hypothesis is Ho: rank(B)=r.
Rejection of the null indicates that the matrix has at least rank=r+1.  In the
standard (stationary) case, the test statistic is distributed as chi-squared
with degrees of freedom = (K-r)*(L-r), where K is the number of Y variables, L
is the number of Z variables, and r is the rank being tested in Ho.  For
example, to test if the matrix is full column rank K where K<L, the null would
be Ho:rank(B)=K-1 and the degrees of freedom of the test would be (K-r)*(L-r) =
(K-(K-1))*(L-(K-1) = (L-K+1).  The default behavior of {cmd:ranktest} is to
perform all possible tests of rank; the {cmd:fullrank} option causes only
the test of whether the matrix is full rank (Ho:r=K-1) to be reported; the
{cmd:nullrank} option causes only the test of whether the matrix is zero
rank (Ho:r=0) to be reported.  

{pstd}The default behavior of {cmd:ranktest} is to report LM tests; the
{cmd:wald} option will cause it to report Wald tests.  P-values are for the
standard (stationary) case using the chi-squared distribution.  Specifying
{cmd:robust}, {opt bw(#)} (where {it:#} is the bandwidth), or
{opt cluster(varname)} will generate an rk statistic that is robust to
heteroskedasticity, autocorrelation, or within-group clustering; {cmd:robust}
combined with {opt bw(#)} will generate a heteroskedasticity and
autocorrelation-consistent (HAC) statistic.  The implementation of an
autocorrelation-consistent statistic and the options available for various
kernels follow that in {helpb ivreg2}; for more details, see Baum, Schaffer,
and Stillman (2007) or {helpb ivreg2} if installed.  If none of the above
options is specified, {cmd:ranktest} defaults to reporting the Anderson
canonical correlations LM test, or if {cmd:wald} is specified, the
Cragg-Donald (1993) Wald test.

{pstd}In the special case of a test for whether a
matrix has rank=zero (e.g., if there is one variable Y), the Anderson,
Cragg-Donald, and Kleibergen-Paap statistics reduce to familiar statistics
available from ordinary least-squares (OLS) estimation.  Thus, if K=1, the
Cragg-Donald Wald statistic can be calculated by regressing the single Y on Z
and X and testing the joint significance of Z using a standard Wald test and a
traditional nonrobust covariance estimator.  The Anderson LM statistic can be
obtained by calculating an LM test of the same joint hypothesis.  The robust
Kleibergen-Paap rk statistics can be obtained by performing the same tests with
the desired robust covariance estimator.  Similarly, if K>1 the test statistics
for rank=0 reported by {cmd:ranktest} can be reproduced by testing the joint
significance of the Z variables across the K equations for the Y variables.
See the examples below.


{marker s_options}{title:Options}

{phang}{cmd:partial(}{it:varlist3}{cmd:)} requests that the variables in
{cmd:(}{it:varlist3}{cmd:)} are partialled-out of the variables in
{cmd:(}{it:varlist1}{cmd:)} and {cmd:(}{it:varlist2}{cmd:)}.  A constant is
automatically partialled-out as well, unless the option {cmd:noconstant} is
specified. 

{phang}{cmd:wald} requests the Wald instead of the LM version of the test.  The
LM version is the default.

{phang}{cmd:allrank} requests that test statistics for rank=0, rank=1, ...,
rank=({it:#}cols-1) be reported, where ({it:#}cols-1) is the number of columns
of the smaller of the two matrices (varlists).  {cmd:allrank} is the
default.

{phang}{cmd:fullrank} requests that only the test statistic for Ho:
rank=(#cols-1) be reported, where (#cols-1) is the number of columns of the
smaller of the two matrices (varlists).  Rejection of the null indicates that
the matrix is of full column rank.

{phang}{cmd:nullrank} requests that only the test statistic for Ho: rank=0
be reported.  Rejection of the null indicates that the matrix has at least
rank=1.

{phang}{cmd:robust} specifies that the Eicker/Huber/White/sandwich
heteroskedastic-robust estimator of variance be used.  The reported rk
statistic will be robust to heteroskedasticity.  

{phang}{cmd:cluster}{cmd:(}{it:varname}{cmd:)} specifies that 
observations are independent across groups (clusters) but not necessarily
independent within groups.  {it:varname} specifies to which group each
observation belongs.  Specifying {cmd:cluster()} implies {cmd:robust}; i.e.,
the reported rk statistic will be robust to both heteroskedasticity and
within-cluster correlation.

{phang}{cmd:bw(}{it:#}{cmd:)} implements autocorrelation-consistent (AC) or
heteroskedasticity- and autocorrelation-consistent (HAC) covariance estimation
with bandwidth equal to {it:#}, where {it:#} is an integer greater than zero.
Specifying {cmd:robust} together with {cmd:bw(}{it:#}{cmd:)} implements HAC
covariance estimation; omitting {cmd:robust} implements AC covariance
estimation.  

{phang}{cmd:kernel(}{it:string}{cmd:)} specifies the kernel to be used for AC
and HAC covariance estimation; the default kernel is Bartlett (also known in
econometrics as Newey-West).  Kernels available are 
{cmdab:bar:tlett}; {cmdab:tru:ncated}; {cmdab:par:zen};
{cmdab:tukey-han:ning} ({cmd:thann} is a synonym);
{cmdab:tukey-ham:ming} ({cmd:thamm} is a synonym);
{cmdab:dan:iell}; {cmdab:ten:t}; and
{cmdab:qua:dratic-spectral} ({cmd:qs} is a synonym).
For some kernels
({cmd:bartlett},
{cmd:parzen},
{cmd:tukey-hanning}, and {cmd:tukey-hamming}) the bandwidth must be at least 2
to obtain an autocorrelation-consistent estimator.  

{phang}{cmd:noconstant} suppresses the constant term (intercept) in the list of
partialled-out variables.

{phang}{cmd:version} displays {cmd:ranktest}'s current version number and 
leaves it in the macro {cmd:s(version)}.  It cannot be used with any other
options.


{marker s_macros}{title:Saved results}

{pstd}{cmd:ranktest} saves the following results in {cmd:r()}:

{synoptset 14 tabbed}{...}
{p2col 5 14 18 2: Scalars}{p_end}
{synopt:{cmd:r(N)}}number of observations{p_end}
{synopt:{cmd:r(N_clust)}}number of clusters{p_end}
{synopt:{cmd:r(chi2)}}rk statistic for highest rank tested{p_end}
{synopt:{cmd:r(p)}}p-value of rk statistic{p_end}
{synopt:{cmd:r(rdf)}}dof of rk statistic{p_end}
{synopt:{cmd:r(rank)}}rank of matrix under Ho for highest rank tested{p_end}

{p2col 5 14 18 2: Macros}{p_end}
{synopt:{cmd:r(version)}}version number of {cmd:ranktest}{p_end}

{p2col 5 14 18 2:Matrices}{p_end}
{synopt:{cmd:r(rkmarix)}}saved results of rank tests{p_end}
{synopt:{cmd:r(ccorr)}}matrix of canonical correlations{p_end}
{synopt:{cmd:r(eval)}}matrix of eigenvalues (=squared canonical correlations){p_end}
{synopt:{cmd:r(V)}}covariance matrix (W in Kleibergen-Paap {bind:[2006, 103]}){p_end}
{p2colreset}


{marker s_examples}{title:Examples}

{pstd}{bf:Tests for underidentification of Klein consumption equation.}

{pmore}Underidentification means endogenous regressors (profits wagetot) are
not identified by the excluded instruments (govt taxnetx year wagegovt capital1
L.totinc) after partialling-out the included instruments (L.totinc _cons).
Test is equivalent to testing whether the matrix of reduced form coefficients
for the endogenous regressors is full rank (#cols=2) vs. less than full rank
(#cols=1).  The test for underidentification should not be confused with a test
for "weak identification"; see e.g. Stock and Yogo (2005) or Baum, Schaffer,
and Stillman (2007).

{phang2}{stata "webuse klein, clear" :. webuse klein, clear}

{phang2}{stata "tsset yr" :. tsset yr}

{pmore}(Klein consumption equation - for reference)

{phang2}{stata "ivreg2 consump L.profits (profits wagetot = govt taxnetx year wagegovt capital1 L.totinc)" :. ivreg2 consump L.profits (profits wagetot = govt taxnetx year wagegovt capital1 L.totinc)}

{pmore}(Homoskedasticity, LM => Anderson canonical correlations test; test all
ranks.  Ho of rank=1 can be rejected, suggesting the model is identified.)

{phang2}{stata "ranktest (profits wagetot) (govt taxnetx year wagegovt capital1 L.totinc), partial(L.profits)" :. ranktest (profits wagetot) (govt taxnetx year wagegovt capital1 L.totinc), partial(L.profits)}

{pmore}(Homoskedasticity, Wald => Cragg-Donald (1993) test; test all ranks.  Ho
of rank=1 can be rejected, suggesting model is identified.)

{phang2}{stata "ranktest (profits wagetot) (govt taxnetx year wagegovt capital1 L.totinc), partial(L.profits) wald" :. ranktest (profits wagetot) (govt taxnetx year wagegovt capital1 L.totinc), partial(L.profits) wald}

{pmore}(Heteroskedastic robust, LM statistic, test for full rank only.  Ho of
rank=1 now cannot be rejected, suggesting model may be underidentified.)

{phang2}{stata "ranktest (profits wagetot) (govt taxnetx year wagegovt capital1 L.totinc), partial(L.profits) full robust" :. ranktest (profits wagetot) (govt taxnetx year wagegovt capital1 L.totinc), partial(L.profits) full robust}

{pmore}(Heteroskedastic and autocorrelation robust, LM statistic, test for null
rank only)

{phang2}{stata "ranktest (profits wagetot) (govt taxnetx year wagegovt capital1 L.totinc), partial(L.profits) null robust bw(2)":. ranktest (profits wagetot) (govt taxnetx year wagegovt capital1 L.totinc), partial(L.profits) null robust bw(2)}


{pstd}{bf:Testing for reduced rank in VAR models.}

{pmore}(Relationship of Johansen trace statistic and Anderson canonical
correlations statistic.  Former is an LR test, {cmd:ranktest} reports LM
version of latter, but based on the same eigenvalues.  The p-values
reported by {cmd:ranktest} are not valid in this application because they are
for the standard stationary case.)

{phang2}{stata "vecrank consump profits wagetot, lags(1)" :. vecrank consump profits wagetot, lags(1)}

{phang2}{stata "ranktest (d.consump d.profits d.wagetot) (L1.consump L1.profits L1.wagetot)" :. ranktest (d.consump d.profits d.wagetot) (L1.consump L1.profits L1.wagetot)}

{phang2}{stata "mat eval=r(eval)" :. mat eval=r(eval)}

{phang2}{stata "mat list eval" :. mat list eval}

{pmore}({cmd:vecrank} LR trace statistic for maximum rank=0 vs. {cmd:ranktest}
LM canonical correlations statistic for same.  Both statistics calculated using
the same eigenvalues.)

{phang2}{stata "di -r(N)*(ln(1-eval[1,1]) + ln(1-eval[1,2]) + ln(1-eval[1,3]))" :. di -r(N)*(ln(1-eval[1,1]) + ln(1-eval[1,2]) + ln(1-eval[1,3]))}

{phang2}{stata "di r(N)*(eval[1,1] + eval[1,2] + eval[1,3])" :. di r(N)*(eval[1,1] + eval[1,2] + eval[1,3])}

{pstd}{bf:Equalities between rk statistic and other test statistics}

{pmore}(Equivalence of rk statistic and canonical correlations under
homoskedasticity)

{phang2}{stata "canon (profits wagetot) (govt taxnetx year wagegovt)" :. canon (profits wagetot) (govt taxnetx year wagegovt)}

{phang2}{stata "mat list e(ccorr)" :. mat list e(ccorr)}

{phang2}{stata "ranktest (profits wagetot) (govt taxnetx year wagegovt)" :. ranktest (profits wagetot) (govt taxnetx year wagegovt)}

{phang2}{stata "mat list r(rkmatrix)" :. mat list r(rkmatrix)}

{pmore}(Equality of rk statistic and Wald test from OLS regression in special
case of single regressor)

{phang2}{stata "ranktest (profits) (govt taxnetx year wagegovt capital1 L.totinc), partial(L.profits) wald robust" :. ranktest (profits) (govt taxnetx year wagegovt capital1 L.totinc), partial(L.profits) wald robust}

{phang2}{stata "regress profits govt taxnetx year wagegovt capital1 L.totinc L.profits, robust" :. regress profits govt taxnetx year wagegovt capital1 L.totinc L.profits, robust}

{phang2}{stata "testparm govt taxnetx year wagegovt capital1 L.totinc" :. testparm govt taxnetx year wagegovt capital1 L.totinc}

{phang2}{stata "di r(F)*r(df)*e(N)/e(df_r)" :. di r(F)*r(df)*e(N)/e(df_r)}

{pmore}(Equality of rk statistic and LM test from OLS regression in special
case of single regressor. Generate a group variable to illustrate
{cmd:cluster()})

{phang2}{stata "gen clustvar = round(yr/2)" :. gen clustvar = round(yr/2)}

{phang2}{stata "ranktest (profits) (govt taxnetx year wagegovt capital1 L.totinc), partial(L.profits) cluster(clustvar)" :. ranktest (profits) (govt taxnetx year wagegovt capital1 L.totinc), partial(L.profits) cluster(clustvar)}

{phang2}{stata "ivreg2 profits L.profits (=govt taxnetx year wagegovt capital1 L.totinc), cluster(clustvar)" :. ivreg2 profits L.profits (=govt taxnetx year wagegovt capital1 L.totinc), cluster(clustvar)}

{phang2}{stata "di e(j)" :. di e(j)}

{pmore}(Equality of rk statistic of null rank and Wald test from OLS
regressions and a Kronecker covariance matrix (independent and homoskedastic
equations).  To show equality, estimate the equations using {cmd:reg3}
specifying that all regressors are exogenous, and then test joint significance
of Z variables in both regressions.  L.profits is the partialled-out variable
and is not tested.)

{phang2}{stata "ranktest (profits wagetot) (govt taxnetx year wagegovt capital1 L.totinc), partial(L.profits) wald null ":. ranktest (profits wagetot) (govt taxnetx year wagegovt capital1 L.totinc), partial(L.profits) wald null}

{phang2}{stata "global e1 (profits govt taxnetx year wagegovt capital1 L.totinc L.profits)" :. global e1 (profits govt taxnetx year wagegovt capital1 L.totinc L.profits)}

{phang2}{stata "global e2 (wagetot govt taxnetx year wagegovt capital1 L.totinc L.profits)" :. global e2 (wagetot govt taxnetx year wagegovt capital1 L.totinc L.profits)}

{phang2}{stata "reg3 $e1 $e2, allexog" :. reg3 $e1 $e2, allexog}

{phang2}{stata "qui test [profits]govt [profits]taxnetx [profits]year [profits]wagegovt [profits]capital1 [profits]L.totinc": . qui test [profits]govt [profits]taxnetx [profits]year [profits]wagegovt [profits]capital1 [profits]L.totinc}

{phang2}{stata "test [wagetot]govt [wagetot]taxnetx [wagetot]year [wagetot]wagegovt [wagetot]capital1 [wagetot]L.totinc, accum": . test [wagetot]govt [wagetot]taxnetx [wagetot]year [wagetot]wagegovt [wagetot]capital1 [wagetot]L.totinc, accum}

{pmore}(Equality of rk statistic of null rank and Wald test from OLS
regressions and {cmd:suest}.  To show equality, use {cmd:suest} to test joint
significance of Z variables in both regressions.  L.profits is the
partialled-out variable and is not tested.   {cmd:suest} introduces a
finite sample adjustment of (N-1)/N.)

{phang2}{stata "ranktest (profits wagetot) (govt taxnetx year wagegovt capital1 L.totinc), partial(L.profits) wald null robust":. ranktest (profits wagetot) (govt taxnetx year wagegovt capital1 L.totinc), partial(L.profits) wald null robust}

{phang2}{stata "di r(chi2)*(r(N)-1)/r(N)": . di r(chi2)*(r(N)-1)/r(N)}

{phang2}{stata "qui regress profits govt taxnetx year wagegovt capital1 L.totinc L.profits":. qui regress profits govt taxnetx year wagegovt capital1 L.totinc L.profits}

{phang2}{stata "est store e1": . est store e1}

{phang2}{stata "qui regress wagetot govt taxnetx year wagegovt capital1 L.totinc L.profits":. qui regress wagetot govt taxnetx year wagegovt capital1 L.totinc L.profits}

{phang2}{stata "est store e2": . est store e2}

{phang2}{stata "qui suest e1 e2": . qui suest e1 e2}

{phang2}{stata "qui test [e1_mean]govt [e1_mean]taxnetx [e1_mean]year [e1_mean]wagegovt [e1_mean]capital1 [e1_mean]L.totinc": . qui test [e1_mean]govt [e1_mean]taxnetx [e1_mean]year [e1_mean]wagegovt [e1_mean]capital1 [e1_mean]L.totinc}

{phang2}{stata "test [e2_mean]govt [e2_mean]taxnetx [e2_mean]year [e2_mean]wagegovt [e2_mean]capital1 [e2_mean]L.totinc, accum": . test [e2_mean]govt [e2_mean]taxnetx [e2_mean]year [e2_mean]wagegovt [e2_mean]capital1 [e2_mean]L.totinc, accum}


{marker s_refs}{title:References}

{phang}Anderson, T. W. 1951. Estimating linear restrictions on regression
coefficients for multivariate normal distributions.
{it:Annals of Mathematical Statistics} 22: 327-351.

{phang}Anderson, T. W. 1984.
{it:Introduction to Multivariate Statistical Analysis}.
2d ed. New York: Wiley.

{phang}Baum, C. F., M. E. Schaffer, and S. Stillman. 2007.
Enhanced routines for instrumental variables/generalized method of moments
estimation and testing. {it:Stata Journal} 7: 465-506.

{phang}Cragg, J. G., and S. G. Donald. 1993. Testing identfiability and
specification in instrumental variables models.
{it:Econometric Theory} 9: 222-240.

{phang}Kleibergen, F., and R. Paap. 2006. Generalized reduced rank tests using
the singular-value decomposition.  {it:Journal of Econometrics} 133: 97-126.

{phang}Stock, J. H., and M. Yogo. 2005. Testing for weak instruments in linear
IV regression. In
{it:Identification and Inference for Econometric Models:}
{it:Essays in Honor of Thomas Rothenberg}, ed.
D. W. K. Andrews and J. H. Stock, 80-108.
Cambridge: Cambridge University Press.


{marker s_acknow}{title:Acknowledgments}

{pstd}We thank Kit Baum and Austin Nichols for helpful suggestions and
feedback.


{title:Authors}

	Frank Kleibergen, Brown University, US
	Frank_Kleibergen@brown.edu

	Mark E Schaffer, Heriot-Watt University, UK
	m.e.schaffer@hw.ac.uk


{marker s_citation}{title:Citation}

{pstd}{cmd:ranktest} is not an official Stata command. It is a free
contribution to the research community, like a paper. Please cite it as such:
{p_end}

{phang}Kleibergen, F., and M. E. Schaffer. 2007.
ranktest: stata module to test the rank of a matrix using the Kleibergen-Paap
rk statistic.  Boston College Department of Economics, Statistical Software
Components S456865.  Downloadable from
{browse "http://ideas.repec.org/c/boc/bocode/s456865.html":http://ideas.repec.org/c/boc/bocode/s456865.html}.{p_end}


{title:Also see}

{psee}Manual:  {hi:[R] canon}{p_end}

{psee}Online:  {helpb canon}, {helpb vecrank}, {helpb ivreg2} (if installed)
{p_end}
